Reading Diary 2

5 Predicting Terms in Hypo-hypernym Relations with Pre-trained T5. terms

annotation

It is the NLP process of adding value to the text by identifying various elements and assigning definitions, meaning and intentions.

Example: The final phase of the process involved the annotation of each example with the correct paraphrase of the relevant MWE [2].

embedding

It is a technique where individual words are represented as real-valued vectors in a lower-dimensional space and captures inter-word semantics.

Example: Probing methods are proposed to determine what information is carried intrinsically by the learned embeddings [3].

labeling

It is the process of adding labels to raw data to give context or meaning to the data.

Example: But each dataset is usually small and scattered because obtaining and labeling such data is time-consuming [3].

pre-training

To first training a model on one task or dataset.

Example: To better model dialogue behavior during pre-training, we incorporate user and system tokens into the masked language modeling [3].

token

It is an instance of a sequence of characters in some particular document that are grouped together as a useful semantic unit for processing.

Example: For Subtask B Pre Train, we introduce single tokens for each MWE in the data [2].